{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "68a8e57f",
   "metadata": {},
   "source": [
    "# Build a Deepfake Detection System \n",
    "\n",
    "This notebook illustrates how to build a video deepfake detection system from scratch using [Towhee](https://towhee.io/). A deepfake detection system predict videos with probabilities being fake ones. This tutorial will use examples from [Deepfake Detection Challenge](https://www.kaggle.com/competitions/deepfake-detection-challenge).\n",
    "\n",
    "Using the sample data of different videos, we will build a basic deepfake detection system within a few lines of code and check the performance using Towhee. At the end, we use [Gradio](https://gradio.app/) to create a showcase that can be played with."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ddf7f23",
   "metadata": {},
   "source": [
    "## Preparation\n",
    "\n",
    "### Install packages\n",
    "\n",
    "Make sure you have installed required python packages:\n",
    "\n",
    "| package |\n",
    "| -- |\n",
    "| towhee |\n",
    "| towhee.models |\n",
    "| dlib |\n",
    "| facenet-pytorch |\n",
    "| albumentations |\n",
    "| timm |\n",
    "| pytorch_toolbelt |\n",
    "| tensorboardx |\n",
    "| tqdm |\n",
    "| gradio |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "96393479",
   "metadata": {},
   "outputs": [],
   "source": [
    "! python -m pip install -q towhee towhee.models dlib facenet-pytorch albumentations timm pytorch_toolbelt tensorboardx tqdm gradio"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8355528c",
   "metadata": {},
   "source": [
    "### Prepare data\n",
    "\n",
    "This tutorial will use a small data extracted from test dataset of [Deepfake Detection Challenge](https://www.kaggle.com/competitions/deepfake-detection-challenge/overview). You can download the subset from [Github](https://github.com/towhee-io/examples/releases/download/data/deepfake_video.zip). This tutorial will just use 2 videos under `test` as example.\n",
    "\n",
    "The data is organized as follows:\n",
    "- **test:** 2 videos from Deepfake Detection Challenge test dataset\n",
    "\n",
    "Let's take a quick look:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ce2ad8ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\n",
      "100 14.3M  100 14.3M    0     0  1592k      0  0:00:09  0:00:09 --:--:-- 3052k\n"
     ]
    }
   ],
   "source": [
    "! curl -L https://github.com/towhee-io/examples/releases/download/data/deepfake_video.zip -O\n",
    "! unzip -q -o deepfake_video.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db28cef1",
   "metadata": {},
   "source": [
    "## Build System\n",
    "\n",
    "Now we are ready to build a deepfake detection system using sample data. We will use the [Combined efficientnet](https://arxiv.org/abs/2107.02612) model to predict the possibilites of being fake ones for input videos. With proper [Towhee operators](https://towhee.io/operators), you don't need to go through video preprocessing & model details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "6047691f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border-collapse: collapse;\"><tr><th style=\"text-align: center; font-size: 130%; border: none;\">scores</th></tr> <tr><td style=\"text-align: center; vertical-align: center; border-right: solid 1px #D3D3D3; border-left: solid 1px #D3D3D3; \">0.99004173</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from towhee import ops, pipe, DataCollection\n",
    "\n",
    "p = (\n",
    "    pipe.input('path')\n",
    "    .map('path', 'scores', ops.towhee.deepfake())\n",
    "    .output('scores')\n",
    ")\n",
    "\n",
    "DataCollection(p('./deepfake_video/test/aagfhgtpmv.mp4')).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8980823c",
   "metadata": {},
   "source": [
    "#### Pipeline Explanation\n",
    "\n",
    "Here are some details for each line of the assemble pipeline:\n",
    "\n",
    "- `towhee.deepfake()`: a Towhee operator applying pretrained models to predict the probabilite a video being a fake one. The higher the score, the higher the probability of it being a fake video.[learn more](https://towhee.io/towhee/deepfake)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77ffadef",
   "metadata": {},
   "source": [
    "## Release a Showcase\n",
    "\n",
    "We've learnt how to build a deepfake detection system. Now it's time to add some interface and release a showcase. We use `deepfake_detection_function` to wrap the pipeline as a [Gradio](https://gradio.app/) application."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "a82a7101",
   "metadata": {},
   "outputs": [],
   "source": [
    "def deepfake_detection_function(path):\n",
    "    return p(path).get_dict()['scores']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "2097730d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7863\n",
      "Running on public URL: https://0fb299f8-a2b2-4257.gradio.live\n",
      "\n",
      "This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"https://0fb299f8-a2b2-4257.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loading state dict /home/xuyu/.towhee/operators/towhee/deepfake/main/weights/final_999_DeepFakeClassifier_tf_efficientnet_b7_ns_0_23\n",
      "loading state dict /home/xuyu/.towhee/operators/towhee/deepfake/main/weights/final_777_DeepFakeClassifier_tf_efficientnet_b7_ns_0_31\n"
     ]
    }
   ],
   "source": [
    "import gradio\n",
    "\n",
    "interface = gradio.Interface(deepfake_detection_function, \n",
    "                             inputs = gradio.Video(source='upload'),\n",
    "                             outputs = [gradio.Textbox(lines=1)]\n",
    "                            )\n",
    "\n",
    "interface.launch(inline=True, share=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3fe8e85",
   "metadata": {},
   "source": [
    "<img src='deepfake.jpg' alt='deepfake_detection_demo' width=700px/>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "ee8f9d005f921b11e37646322d569d83ab1bb8b2f1f9e1244f064a47f10136b5"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
